Understanding the limitations of retrospective analyses of poison center data.

Limitations of retrospective analyses R.S. Hoffman Poison centers provide poison prevention and safety information, professional education, and assistance with the diagnosis and treatment of poisoning. These invaluable efforts are linked to medically and legally mandated record keeping, which entails continuous and archived collection of data over time. Surrounded by gigabytes of information regarding exposures to a vast array of known and potential toxins, poison center researchers are understandably motivated to study these data. Additionally, there is a general need to participate in academic endeavors, an altruistic desire to change or improve healthcare, reduce unnecessary healthcare expenditures, and a survival advantage to justifying poison centers as valued participants in the public health arena. These powerful stimuli to analyze readily available poison center records have clearly been facilitated by significant advances in computerization that have occurred over the last few decades. Now, in minutes, with little advanced preparation or financial outlay, any poison center researcher can gather and analyze thousands of records. Reports of poison center data appear frequently in medical journals (1–3), are often incorporated into triage and management guidelines (4–6) and occasionally are used to address consequential public health issues (7). As these studies become more prevalent, it is essential that we identify and acknowledge the strengths, weaknesses, and inherent biases associated with this methodology. A major strength is that the data are already being collected and are stored in a form that is both easy and inexpensive to search. Another strength is the ability to easily gather a very large sample size of cases of a specific exposure. Analysis of the descriptive epidemiology of the cases and trends over time can inform both clinicians and policy makers. Large case numbers lend weight to such analysis, whether or not such confidence is justified. However, we must accept that a retrospective analysis of data collected for a purpose other than the specific study intent has methodological limitations. The data may be insufficient to answer the study question, incomplete, or frankly inaccurate (8). As an example, the presence or extent of symptoms may not be completely recorded; when no symptoms are listed, either the patient was truly asymptomatic, or the charting was incomplete. Similarly, while doses may be overestimated to create worst case scenarios that help provide conservative medical care, this clearly biases subsequent research questions. Prospective hypothesis-driven studies of poison center data can improve these shortcomings if the data collection instrument is changed to a more complete, detailed and accurate template. Discrepancies between reported and non-reported cases, differences between hospital and poison center records on the same case and some other biases and limitations in poison center data are discussed elsewhere (9–11). Two articles in the current issue of Clinical Toxicology highlight these concerns. The first article reports on 582 clopidogrel exposures collected over several years (12). Although the authors acknowledge that the exact dose was not collected “in a consistent fashion,” they later state that the dose was “determined in 344”of these exposures and report a mean dose of 249 mg [emphasis added]. Despite only obtaining a final medical outcome in 49% of cases and having a large percentage of patients with exposures to additional substances, they conclude that medical outcomes are generally good and that these data can be used for education and prevention strategies. A poison center could create a triage guideline for clopidogrel exposures based on this type of data. One could argue, however, that with the exception of 7 patients who allegedly ingested clopidogrel alone and developed symptoms, it is unclear that any of the remaining patients actually ingested the drug. The second relevant article in this issue discusses exactly this concern and introduces us to a term called the “unproven ingestion” (13). Children with clinically worrisome methanol or ethylene glycol exposures were referred by the poison center to hospitals. A unique situation existed where the poison center funded the analysis, which resulted in actual testing in 102 of 115 cases. Only 21 of the 102 children tested had detectable levels. Not all of these 21 children were symptomatic, and some children with symptoms had negative levels (presumably from their other ingestions, or unrelated medical issues). The authors correctly conclude that an unproven ingestion (with or without symptoms) has the potential to introduce a significant bias in the data set and conclude that laboratory confirmation of the exposure is essential. This is a lesson that most of us have learned clinically at the bedside of a patient who swears that they have ingested an entire bottle of acetaminophen (or anything else) only to have a non-existent or trivial level. Adults, adolescents, and sometimes younger children intentionally deceive us in a poison exposure event, and parents and poison specialists often overestimate ingestions to provide good conservative care. Received 19 January 2007; accepted 22 January 2007. Address correspondence to Robert S. Hoffman, New York City Poison Center, 455 First Ave., Room 123, New York, NY 10016. E-mail: [email protected] C lin ic al T ox ic ol og y D ow nl oa de d fr om in fo rm ah ea lth ca re .c om b y 67 .1 86 .9 6. 95 o n 05 /0 5/ 12